cn 
o 
o 

(N 
o 

Q 

(N 

H 



Nonparametric Estimation in the Model of Moving Average. 



^ ' Alexander Alekseev 

O ■ February 1, 2008 

(N 

m 
o 

B 

> 

X 



1 Introduction. 



The subject of robust estimation in time series is widely discussed in literature. One of the approach- 
es is to use GM-estimation. This method incorporates a broad class of nonparametric estimators 
which under suitable conditions includes estimators robust to outliers in data. For the linear models 
the sensitivity of GM-estimators to outliers have been studied in the work by Martin and Yohai 
[5], and influence functionals for this estimator were derived. In this paper we follow this direction 
and examine the asymptotical properties of the class of M-estimators, which is narrower than the 
class of GM-estimators, but gives more insight into asymptotical properties of such estimators. 
This paper gives an asymptotic expansion of the residual weighted empirical process, which allows 
to prove asymptotic normality of these estimators in case of non-smooth objective functions. For 
simplicity MA(1) model is considered, but it will be shown that even in this case mathematical 
techniques used to derive these asymptotic properties appear to be rather complicated.However, 
the approach used in this paper could be applied to GM-estimators and to more realistic models. 



2 Main Results. 



In this work we consider the model of moving average MA(1): 

(1) Ui = ei-a£i-i, i = 0, ±1,±2, . . . , 

where {si} - iid, Eei = 0, Ee^ < oo, \a\ < 1. 

Let Ui,...,Un be the observations of a random variable u. 

For every 9 G R-*^ set 

eo{0) = 0, 



(2) 

It can easily be seen that 

(3) 



£i{e) = ui + e£i-i{9), 



1,2, 



3=0 

where £i{9) are the residuals of the model (1). Assume for a moment that the equation of a moving 
average holds only for i = 1,2, .. . with = O.AIso let's assume that variables {e^, i > 1} have 
a density function g{x). Then the maximum likelihood equation for the estimation of a can be 



(ui, . . . 
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= (ei,- 




)^, and the matrix J(a) as 
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i-1 

Then the equations £j = a^Ui-j, i = 1, . . . ,n can be rewritten as: 

j=0 

£ = J{a)U 

li fuivi, ■ ■ ■ ,yn) is the density function of the vector U, then 

n / i—1 

fu{yi, ...,yn) = Yl9 [Yl '^^Vi-j 

i=l \i=0 

Therefore, the maximum hkehhood estimator, which is defined as a solution of the maximization 
problem 

log/w(yi,--->yn) — ^•sup, 

d 

can be obtained as a root of the equation 

U 9{e^{e)) ' 

In this paper we examine a natural generalization of this estimator for the model(l), namely, M- 

estimator S„ of the parameter a. 

The estimator a„is defined as a solution of the equation 

1=1 

with ei{9), determined by (2) or (3), where is an a priori known function, which we will choose 
later. 

The asymptotic distribution of the estimator a„ in the case when the function ^(•) satisfies 
Var|^^[^'] < oo will be derived. This result will be obtained with the help of asymptotically 
uniform expansion of the residual weighted empirical process, which will be defined later and which 
will be of interest by its own. 



We fix some notation: 

fc=i fc=i 

Un{x, 61) := i V ^^3^ l{£k{0) < x), Unix, a) := - V l{£k < x), 

k=l k=l 



2 



where Si are derived from (1): 

£i = ^ ^ OpUi—j. 

i>o 

If the variation of ^ is bounded,the following is true: 



n 



1/2 



ln{ot + n "^/^r) — Z„(q;) = / ^{x)dn}'^'^[un{x,a + n ^^^r) — u„(a;, a)] 



+00 



= — J n^^'^[un{x,a + n ^^"^t) — Un{x,a)]d^(x) + ^n^^'^[un{x,a + n ^^"^t) — Un{x,a)]^{x)^ 

—00 

It will be proved below that the second term is equal to zero. Now consider the first term. 
Theorem 1. Assume that the following conditions hold: 

E{eif < oo; 



+00 
—00 



Then 



sup 

a;eR,|x|<0 



g{x) > 0, lim g{x) = 0, sup < 00. 



n^/'^[un{x, a + n ^/^r) - Unix, a)] + Tg{x) 

1 — 



= Op(l). 



The first theorem will be proved in section 2. The next theorem is the main result of the work 
and its proof utilizes the first theorem. 



Theorem 2. Assume that the following conditions hold: 
(i) E{eif < 00, 

g{x) > 0, lim g{x) = 0, lim g{x) 



0, sup \g'{x)\ < 00; 



(ii) 



Var|i~ [^]<oo, 
gd^ ^ 0, 
misi)] = 0. 



then 

1) if ^ is continuous on Ti} , then with probability tending to one, there exists a n^^^ -consistent 
solution of the following equation 



n 



(4) n-5:^*(e.(^)) = 0; 

k=l 

2) for any n)-!'^ - consistent solution OLn of (4) the following statement 
holds 

where 

ai{a) = (1 - a^)- ^ . 



/ gd^ E{ei) 



2 



The function ^(x) = F{x) — ^ where F{x) is a continuous distribution function of a certain 
zero mean symmetrical distribution satisfies the conditions of theorem 2 in case £j are symmetrical 
zero mean random variables. 



3 Proofs of the theorems 



Lemma 1. If for some p>l -E'|£i|*' < oo, then: 

dek{a) 



Proof. 



E 



dek{a) 



de 



sup E 

l<k<n 



sup E 

l<k<n 



de 



a) 



89' 



< oo; 



< oo. 



' k-l 



' k-1 



< E I ^j|arVfe-i| < cE | < 



4 



'fc-1 



k-1 



< cE ^a{|£fe_i| + |a| J]aj|£fe_i_j| < 



< 2c 



' k-l 



i=i 



'k-l 



where ai G [0, +00) : 



E I ^a{\£k-i\ + |a|E ^a{\ek-i- 
i=i / \j=i 



Ictl < Q!l < 1 

, . Therefore, it is sufficient to prove 



2\aY < coc 



'k-l 



sup Ie [ ^a{\ek-j\ > < 00. 



l<it<n 



This clear from the Minkovsky's inequality: 

/k-l \py/p r \P>ji/p 

what proves the first claim of the lemma. The second one can be proved in the similar way. 



Lemma 2. set 



k-l 



t>k 



t=0 



£0 



if E{ei)^ < 00, then there exists such a ak, that 



sup |afc(r)| < dk, 
\r\<e 

sup E{dk)^ < 00. 

l<A;<n 



Proof. Let B G (0, 1) : \a\ + 9n~^/^ < S forn > no, < 6' < 00, then (7jt(r) < ak, where 



ajk := 0n-V2 ^ + ^'^-^ ^ tB'-^\ek-i-t\ - 



£0- 



5 



£0 



t>fc 



t>i 



For Va, 6 > 0,p > 1 we have the following inequality {a + by < IP ^{dP ^\P). Hence, it follows that 



t>l 



By Minkovsky's inequality: 



1/4 



+ 



+ < 

hence, 



E idn-^l'' J2 B'\ek-i-t\] ] < {e'n-' [b'' + 3^'^+'^ + . . . ) ^{8,)'}'^' < 00, 



EUn-V2^B*|£fe_i_t| <oo. 



1/4 
> < 



01 



< {^«n-^ (1 • 5° + 2 • + ■ ■ ■ + + . . . ) E(£i)4 - B^Eiei)"^^ 



1/4 



Since the series ^ ti? converges, it follows that 
t>i 



t>i 



what proves the lemma. 



Lemma 3. Let the following conditions hold: 

(i) E{eif < 00 



(a) g{x) is the density function of £i that satisfies the following conditions: 

lim gix) = 0, 
+ oo 

sup < OO. 

X 

For zin{x,T) defined as 



the following holds: 



\zin{x,T)\ = Op(l). 



Proof. 



Zln{x 



Mr>'^ ( deuia + n ^/^r) dek{a)\ / , 1/9 n \ 



it=i 



-I (^efc(a + n-^/V) < xj + 
l(^£fe(a + n-^/V) <x) = 



rn 



fc=i 



where ^ G [a, a + n ^/^r]. Therefore 



\zin{x,T)\ < 



TU 



+ 



fe=i 



rn 



(a + n"^/V 



-) < - I {£k < x) 



+ 



k=l 



+ Op(l). 



To transform the right hand of this inequahty, we introduce the random process L^{x,t): 

n 

L+{x, r) := rn"^ ^ a+ [l (^ek{a + n'^/V) < - I (£fe < a;)] , 



k=l 



where 



- \ 0, if ^ < 0. 

n 

L~{x,t) := Tn"^^a^ I ^eA;(a + n"^/^r) < a;) 



ik=l 



0, if ^>0. 



Til 



fe=i 



<|L+(x,r)| + |L-(x,r)|. 



We first show that L+(x,r) = op(l). The proof utihzes the expansion obtained in lemma 7.1 [1] 

£fe(a + rn~^/^) = £fe + ttT'^/'^ n^_^ + crfe(r), 

and lemma 2. 



rn 



n 



fc=i 



< 



< 



n 

rn~^ E flfe (a) I (^fc < a; - rn'^^'^ tJ,j,_i + dk^ - l(£fe < x) 



fe=i 



Consider random processes Vn{x,T),Vn{x): 



n 

Vnix, r) := ^ (sk < x - tu^^/"^ ji^-i + - G (^x - Tn~^^^n^,_i + 



ifc=i 



Vnix) :=n [I i£k < x) - G (x)] . 



k=l 



then 



n 

Tn~^ E flfc (a) l(£k <x- rn"^/^/Xfc_i + ct^) - < 



X 



k=l 



< 



< \T{Vn{x,T) - Vn{x))\ + 



n 



n 



k=l 



8 



and 



n 



n 

' E (C {x - rn-^'''l^k-i + ^k) - G{x)) 



fe=i 



n 



-3/2 



Op(l), 



fc=i fe=i 

where ^ € [x — Tn~^/'^ + ak,x\. To prove that |v„(x,r) — Vn{x)\ = Op(l) we use theorem 2.1 
from [2]. Let's check whether the conditions of this theorem . The condition E|£i|^ < oo imphes 
E|£i|^ < oo, and because of the lemmal the latter yields 



sup E 

l<fe<n 



< oo. 



Hence, 



n 



fc=l 



4 1 

To check the condition n"^ Yl E {a^{a)) = 0(1) we use Cauchy-Bunyakovskii inequality: 

k=i '- ^ 



1/2 



k=l 



because of the lemmal and lemma2: 



k=l 



sup E(a^(a)) < oo, 

l<fe<n 

sup rL/CTj^. < oo, 

l<fc<n 



therefore. 



n-'^E[{at{a))'al\=Oil). 

k=l 

Now we check the following condition: 

n 

n-'Y.at{a)al = Opil). 



k=l 



9 



Lemma 2 yields sup EBf, < oo, 

l<fe<n 



n 



'^^(^ki^)^k < sup al]n ^^a^{a) = sup U ^ ^ 



fe=l 



ik=l 



fe=l 



Consider a random process 



92 



e(a) 



89' 



Y,{3{j-^)wr''\^-o\) 

j=2 



then 



E 



52£(q) 



< 2c' 



yj = fe+l 

2n 



E oPi\e-j\\ +E |q| 



j=fc+i 



2c2Eef 



j=k+l j=k+l 



• 0, A; — > 00, 



where ai : 
Hence 



and 



|a| < «! < 1 
i(i-l)|ap-2<cai 



n 



fe=i 



9^6^ (a) 



■E|^j(i-l)|ar^| 



fe=i 



So all the conditions of the theorem 2.1 from [2] hold, therefore 

sup n^/2 [i;n(aJ,r) - Vn{x)] = Op(l). 

3;GR^,|t|<0 

we have proved that 

n 



Op(l), 



10 



and 

Therefore, 



-1 d'^^ki 
rn > ^ 



a 



|L+(x,r)| = Op(l). 



I (ek{a + tT^I'^t) <x \ - \{ek<x 



Op(l). 



To finish the proof of the lemma we need to show that 

.1 92efc(a) 



rn 



fc=i 



T (ejk < x) 



Op(l). 



rn 



fe=i 



< 



fe=i 



+ 



rn 



We apply theorem 2.1 from [2] to process ^ [l (sfe < a^) — G^l^;)] 

In the same way it can be shown that 



k=l 



n 



k=l \j=2 



0. 



This finishes the proof of the lemma. 



Lemma 4. Assume that the following conditions hold: 
(i) E{eif < oo; 



(u). 
Let 

then 



g{x) > 0, lim g{x) = 0, sup |5''(a;)| < co 



Z2nix, r) := n-^/2 ^ Ji (^Skia + n-^/V) < - I (£jk < x) 



k=l 



Z2n{x,r) + Tg{x)- 2^^i 

1 — 



Op(l). 
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Proof. 



k=i 



consider a process i;„(x,r): 



t;„(x,r) := n 
then 



jk=l 



5^ 



-1/2, 



G(x-rn-V2^fe_i-afc(r))-G(ar)' 



fe=i 



By analogy to lemma 3 it can be shown that 



sup n^/^ [Vn{x,T) - Vn{x)] = Op(l). 

xe'R},\T\<e 



Consider a process 



n 



-1/2 ■ 



k=l 



dek{a) 



n ^'"^ — ^rrr— I -rn 



fe=i 



9^ 



k=l 



-Tg{x)E < 

where ^ G [x — Tn~^/^/Xj._i — c7jk(T),x], and 



oo \ / oo 

j=i / \j=o 



1 — a 



what proves the lemma. 



12 



Proof of Theorem 1 

Rewrite the process in the following way: 

n^^'^[un{x, a + n^-^/^r) - «„(x, a)] = zin{x, r) + Z2n{x, r), 

where 

I (^Sk{a + n~^/V) <x^ -I{ek<x) . 

k=l 

Now it can be seen that lemma 3 and lemma4 prove the theorem. 



Z2n{x,T) := n ^/^^ 



80 



Proof of Theorem 2 



1) 



A(a) = - I 



l-a2' 



k=l 



89 

k=l 



Without loss of generality we consider A(a) > 0, such that there exists A> satisfying 

(2) n^/\ia + n-^/^A) = n^l%{a) + \{a)A + op(l) > 0; 



(3) 



n 



^/2/n(a - n-^/^A) = r?'Hn{a) - \{a)A + op(l) < 0. 



Since ln{0) is continuous on 9 in some neighbourhood of a, then because of (2) and (3), a 

consistent solution of (l)exists in interval (q — n~^^'^A\a + n~^/'^A). 

2) 



n 



1/2 



n 



1/2 



+00 



13 



Since Var [^(a:^)] < oo holds , and because of theorem 1 and conditions imposed on g{x), it can 
be obtained that {n-*^/^ [unix, a + n~^/'^T) — Un{x, a)] *(a;)}^^ = 0, and also that 



n 



1/2 



oo 
— oo 

+ 0O 



uniformly on |r| < G. 

Let an be a n^/^-consistent solution of (1), that is n^/^(S„ — a) = Op(l). Since 



then 



Consider the process 



If we denote 



n^^Xian) = n^^Xia) + X{a)n^/\an - a) + Op(l) = 0, 
nV2(3„ - a) = -[A(a)]-^ • n^/%{a) + op(l). 

k=l 



then 



Uk- 



< ca'l 



for some ai, 1 > ai > lal. 



E 



n 



-1/2 



it=i 



j=fe+i 



de 



de 

2>i 1/2 



^{Sk) 



< 



de ae 

1/2 



< cn-V2 ^ c^fc {E*2(£,)}V^ ^ c^-i/2 (E*2(£^)) ^ = o(l). 



(*) 



k=l 



k=l 



14 



9s ici] 

Since ^f. := — — — ^'(efc) is strictly stationary and forms a martingale-difference, then the central 
limit theorem can be applied to it: 



11 



-1/2 



k=l 



Because of the proved property (*) we obtain 



11 



i/\(a)^N I 0,E I I •E*2(e,) 



nV2(3„ -a) = -[A(a)]-i ■ n^Hnia) + Op(l) ^ N 



/ /oo \^ \ 



0, 



(l-a2)2 



Since 



then 



(7^(a) = (l-a") 



Eef 



2, ¥.^\ei) 



2 ' 



what proves theorem 2. 
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